Learning Objectives

After completing this lesson, you’ll be able to:

Note

Learning content in the FME Academy presents a user's story addressing their data integration challenges with FME. You should follow along with their actions using your own installation of FME (2023.0 or later) or request an on-demand virtual machine in the footer link below. Some lessons will require you to follow their steps or take additional steps to answer a quiz question.

The Resources sections provide links to interactive tutorials and starting workspaces when necessary.

Resources

Video

Note

Videos are provided in this course if you prefer to watch instead of reading the text below. Note that some Quiz answers might require you to read the text.

Automate Repetitive Data Integration Tasks with FME Flow

Frank

Frank is a GIS Administrator working with Jennifer for a local government. He is also the FME Flow Administrator for his organization. Because his town doesn't have an open data portal, he is constantly bombarded with emails from other departments and the public asking for GIS data. He has to read through the emails, find out what layer people are requesting, and manually send it to them. People most often request layers from the community resources geodatabase, which contains data about street food vendors, parks, community centers, etc.

His colleague Jennifer offers to help him use FME Flow to let users access the data directly without his help, creating self-serve data delivery.

Note

FME Flow can automatically run workspaces on a schedule or in response to a trigger such as receiving an email. It also lets you manage data securely and make data accessible to everyone in your organization.

Note

FME Flow has five default security roles. You will need at least the fmeauthor role to complete this course.

Learn More

Give Users Control Using Published Parameters

Jennifer sends Frank a workspace to help him get started. He downloads and opens it with FME Workbench. 

Note

Follow along with Frank's steps using your own version of FME.

It’s a simple workspace that reads in all the feature classes of an Esri geodatabase and then writes them out in the format of a user’s choice.

Starting workspace

Jennifer tells him he can create a workspace that gives end-users control over what data they get. She suggests two steps to let users customize the data they receive:

  1. Let them choose the format
  2. Let them choose the layers

Note

Jennifer’s advice raises an important point. Remember to design your workspaces based on the FME platform user roles. Frank has to design his workspace so it works well for end-users. These end-users might be running the workspace as logged-in users of FME Flow or using a public-facing FME Flow App. The choices he provides need to make sense for these situations.

The first step is to let users specify which data format they want by providing them with a fixed selection of format choices. Since the workspace uses a Generic writer, users can choose any format. This unlimited choice can lead to problems:

Jennifer tells Frank he can create user parameters to address this issue. User parameters give users control over how a workspace runs. You can use published parameters when running workspaces locally, but it is critical to use them when creating workspaces for FME Flow, which are likely to be run by other people.

Note

User parameters can be published or private.

Define Which Formats Users Can Write To

To restrict the formats available for writing, Frank finds the Training [GENERIC] writer in the Navigator. He clicks the drop-down arrow and expands the Parameters section.

Output format user parameter in the Navigator

He double-clicks the Output Format parameter and sees it’s currently assigned to “Esri Shapefile.”

Esri Shapefile default value

Frank sees that the output format is linked to a user parameter in the Navigator window because it has a different icon and says “(Linked to ‘FORMAT_GENERIC’).” Most parameters in a workspace – reader, writer, and transformer – can be linked to published parameters to let users set their value when they run the workspace.

Frank wants to see how the parameter works, so he clicks the Run menu, ensures Prompt for User Parameters is checked, and then clicks Run.

Enabling Prompt for User Parameters

The Translation Parameter Values dialog asks Frank to select a value for Output Format, defaulting to “Esri Shapefile.” Frank clicks the drop-down menu for Output Format and sees he can pick any format supported by FME. That's too many options.

Default behavior: pick any format

To fix this problem, Frank closes the dialog and looks at the User Parameters section of the Navigator. He finds the “[FORMAT_GENERIC]” published parameter. He will replace this to restrict the user’s options. He right-clicks it and selects Delete.

Deleting a user parameter

Then he right-clicks User Parameters and clicks Manage User Parameters...

Manage User Parameters in the Navigator

Note

Frank is deleting the existing user parameter and adding a new one because some user parameters that are created automatically cannot be edited fully. For those, it’s better to start fresh.

The Parameter Manager dialog opens. Frank clicks the Insert button (the green plus) to add a user parameter. There are many types of user parameters available. In this case, Frank chooses Choice, which lets him provide a list of options to the user that map onto a different value provided to the workspace. This user parameter is useful for letting users choose formats or coordinate systems because it hides the more complex name FME needs and instead shows a simple version to the user.

Adding a Choice user parameter

On the right-hand side, Frank fills out the dialog with the following parameter properties:

Parameter Identifier OutputFormat
Prompt Enter an output format
Published Enabled
Required Disabled
Disable Attribute Assignment Enabled
Choice Configuration Dropdown

To fill in the Choices table, rather than manually filling in the cells in the Value and Display columns, Frank wants to import the file formats the user can choose from. He clicks Import Writer Formats.

Importing writer formats to fill in the Choices table

In the Select Writer Formats dialog, use the Search bar to search for the following six formats. Click the check to add them to the list.

Selecting writer formats

Click OK.

The selected formats appear in the Choices table:

Populated Choices table

Frank uses a table here to show the user a more readable name for the format (Display = Description) instead of the value that FME needs to choose (Value = Short Name).

Finally, Frank selects Esri Shapefile as the default value.

Selecting default and clicking OK to create the user parameter

He clicks OK to create the published parameter.

Now that he has created a published parameter, he needs to link it to the correct writer parameter for it to work. He returns to the writer’s Output Format parameter, right-clicks it, and clicks Link to User Parameter.

Linking a user parameter to an FME parameter in the Navigator

In the Set to User Parameter dialog, he picks OutputFormat from the list and clicks OK.

Selecting the user parameter to link

When he clicks Run, he is prompted to choose an output format from the restricted list. The workspace runs, and he receives the data in his chosen format.

Selecting an output format

Exercise

Frank is happy with the format selection, but now he’d like to let users pick the layers from the geodatabase they receive. He knows he can control this with the user parameter Feature Types to Read, found in the Navigator under CommunityMap [FILEGDB] reader > Parameters Features to Read.

For the exercise, create a new published parameter that lets the user pick which feature types to read and write. Hint: in some cases, creating a published parameter is as easy as right-clicking the parameter you want to link and selecting Create User Parameter.